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Preface 


It  is  my  great  pleasure  to  welcome  you  all  to  the  Brain  Science  Research  Center  at  KAIST  in  Taejon.  We 
join  together  to  hold  the  second  Korea-US  Joint  Workshop  on  Brain  Science  -  Cognitive  and  Behavioral 
Neuroscience.  We  expect  this  workshop  provide  the  opportunities  to  understand  different  areas  in  brain 
science,  and  to  promote  the  bilateral  cooperation  between  Korea  and  US  researchers.  It  must  be  exciting 
to  explore  the  mysteries  of  brain,  and  it  is  a  critical  issue  how  brain  enables  mind.  These  issues  and 
mysteries  are  broadly  discussed  in  this  workshop.  The  theme  of  the  second  workshop  covers  high 
cognitive  brain  functions  such  as  single  cell  recording  and  brain  imaging  studies,  selective  attention,  and 
language. 

I  would  like  to  thank  the  organizing  committee,  Drs.  Hemin  Chin,  Yong  Shin,  ChanSub  Chung,  Jung-Mo 
Lee,  and  Choongkil  Lee.  Our  thanks  also  go  to  the  invited  speakers  for  their  sharing  their  exciting  results 
with  us.  I  also  would  like  to  thank  the  BSRC  researchers  and  staffs  for  their  hard  work  in  organizing  this 
workshop. 

Thanks  are  also  due  to  our  sponsors,  i.e.,  Korean  Ministry  of  Science  and  Technology  and  Asian  Office  of 
Aerospace  Research  &  Development,  US  Air  Force  Office  of  Scientific  Researches.  Thank  you  all  for 
your  participation. 

January  2000 


Soo- Young  Lee 

Director,  Brain  Science  Research  Center  (BSRC) 

Korea  Advanced  Institute  of  Science  and  Technology  (KAIST) 
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Temporary  Deafferentation  -  Induced  Changes  of  Neural  Networks 
in  the  SI  Cortex  of  Rats 

Hyung-Cheul  Shin 

Department  of  Physiology,  College  of  Medicine,  Hallym  University 

We  have  carried  out  simultaneous  many  (up  to  15)  single  unit  recordings  from  the  primary 
somatosensory  (SI)  cortex  of  anesthetized  rats  before  and  after  temporary  deafferentation  (TD)  to  the 
receptive  field  (RF,  forepaw  digits)  by  the  injection  of  lidocaine  (1-2%).  The  blocking  of  afferent 
transmission  and  the  emergence  of  new  RF  were  quantitatively  determined  by  the  analysis  of  the 
post-stimulus  time  histograms.  Responsiveness  to  the  RF  stimulation  was  diminished  by  86.44  +  3.5% 
at  10  min  after  lidocaine  injection  to  the  RF  center,  while  it  was  augmented  by  21.58+  9.5%  at  10 
min  after  TD  to  the  outside  of  the  RF.  Changes  of  functional  connectivity  patterns  among 
simultaneously  recorded  neurons  were  determined  by  analyzing  spike-triggered  cross-correlograms. 
Co-firing  patterns  were  frequently  observed  either  from  spontaneously  active  neurons  or  from 
peripherally  activated  neurons  and  they  were  usually  diminished  and  then  recovered  during  TD.  Cells 
showing  less  strong  responsiveness  to  the  RF  activation  received  both  strong  excitatory  inputs  from 
RF  center  cells  and  weak  excitations  form  RF  boundary  cells.  During  TD,  inputs  from  RF  center  cells 
were  reduced  and  then  recovered,  while  weak  excitations  from  RF  boundary  cells  became  stronger 
and  then  weakened.  These  results  provide  evidences  that  the  emergence  of  new  RF  after  TD  may 
actually  involve  the  activation  of  originally  weak,  latent  connections,  (supported  by  a  '98  grant  to 
HCShin  from  the  Korean  Ministry  of  Science  &  Engineering  and  the  Brain  Science  Research 
Program). 

Introduction 

It  is  well  known  that  peripheral  sensory  deprivation  (deafferentation)  induces  receptive  field  (RF)  changes 
and  consequent  reorganization  of  the  body  representation  (somatotopy)  within  the  centreal  somatosensory 
system  of  adult  animals  after  pathological/experimental  lesions  at  various  levels  of  this  system  (Nicolelis 
et  al.,  1993;  Faggin  et  al.,  1997).  Reversible  changes  RF  (such  as  changes  of  RF  size  or  appearnce  of  new 
RF  (NRF))  of  somatosensory  neurons  can  occur  rapidly,  within  minutes,  in  response  to  temporary  RF 
lesions  (Merzenich  et  al.,  1983;  Calford  and  Tweedale,  1988;  Donoghue  et  al.,  1990).  This  somatosensory 
plasticity  involves:  1)  increase  of  the  excitablility  of  neurons,  2)  disinhibition  of  lateral  inhibition  and  3) 
changes  of  synaptic  strength  among  neurons.  Thus,  there  is  a  strong  possibility  of  the  dynamic  changes 
neuronal  networks  among  somatosensory  neurons  during  temporary  deafferentation  (TD)  of  peripheral 
RF.  However,  there  has  been  no  systematic  study  to  verify  this  possibility. 


Purpose 

Simultaneous  many-single  unit  recordings  from  the  prumary  somatosensory  (SI)  cortex  and  the  ventral 
poserior  lateral  (VPL)  thalamus  of  anesthetized  rats  were  done  to  characterize  the  changes  of  neural 
networks  among  somatosensory  neurons  following  temporary  deafferentation  (TD)  of  local  anesthesia  to 
the  RF  (forepaw). 


Methods 

Adult  Sprague-Dawley  rats  (200-300g  n=10)  were  anesthetized  with  urethane  (i.p.  20%,  7ml/kg).  For 
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single  unit  recording,  craniotomy  was  done  over  the  forepaw  area  of  the  SI  cortex  (0-1  mm  rostral  from 
bregma,  3-5mm  lateral  from  midline).  Tungsten  wire  (diameter  50p  m,  teflon-coated,  4X2  array) 
electrodes  were  driven  to  the  layer  IV  of  the  SI  cortex.  Bipolar  concentric  stimulating  electrode  (1Hz, 
1ms  duration,  1.5-3V)  was  subcutaneously  inserted  under  the  RF  (forepaw  digit)  to  activate  periperal 
sensors.  Isolation  of  individual  units  was  done  by  using  spike  sorting  function  of  the  NEX  program.  After 
30  min  of  neural  recording,  2%  lidocaine  (30  p  1)  was  injected  under  the  peripheral  RF,  s.c.,  to  induce 
temporary  deafferentation  (TD).  Post-stimulus  time  histogram  (PSTH)  was  analyzed  to  quantiatate 
neuronal  responsiveness  to  the  peripheral  stimulation  and  spike-triggered  crosscorrelation  analysis  was 
carried  out  to  examine  any  changes  of  neural  connectivities  among  neurons. 

Results  and  Discussion 

Evoked  unit  responses  of  SI  cortex  and  VPL  thalamic  neurons  were  facilitated  by  the  local  anesthetic 
deafferentation  (TD)  in  the  RF  boundary,  but  they  were  suppressed  by  TD  in  the  RF  center.  This  suggests 
that  the  presence  of  lateral  suppressions  among  neurons  (Smith  and  Gilbert,  1994).  We  have  also  shown 
TD-induced  disappearnce  of  the  lateral  suppressions  from  RF  center  cells  to  the  RF  boundary  cells.  It  has 
been  reported  that  the  blockade  of  GABAergic  transmission  caused  the  expansion  of  neuronal  RF  (Land 
et  al.,  1995). 

TD-induced  abolition  or  weakening  of  the  cofiring  activities  among  RF  center  cells  were  frequently 
observed  in  this  study.  These  RF  center  cells  were  excitated  by  unknown  common  inputs(s),  which  may 
be  thalamocortical  afferents  (Jensen  and  Killackey,  1987).  However,  TD  enhanced  the  cofiring  activities 
among  RF  boundary  cells.  Present  study  also  showed  the  appearance  of  newly  firing  neurons  and  new 
crosscorrelation  among  neurons  following  TD  to  the  RF  center.  TD  strengthened  the  originally  weak 
excitatory  inputs  from  the  RF  boundary  cells  to  the  RF  center  cells.  This  may  explain  the  previously 
reported  RF  expansion  to  the  RF  neighbour  after  TD  to  the  RF  center  (Calford  &  Teedale,  1988).  We  have 
also  noticed  that  a  newly  firing  cell  was  activated  by  a  RF  boundary  cell  after  TD.  This  emergence  of  new 
excitatory  connections  from  RF  boundary  to  RF  outside  cell  may  be  responsible  for  the  emergence  of  the 
NRF.  During  TD,  occurrence  of  new  firing  and  new  excitatory  connections  were  more  frequently 
observed  in  the  SI  cortex  than  in  the  VPL  thalamus.  This  indicates  that  SI  cortex  is  more  plastic  than  VPL 
thalamus.  In  this  study,  corticothalamic  components  were  faciliated,  while  thalamocortical  ascending 
transmissions  were  suppressed  by  TD  in  the  RF  center  cells,  suggesting  thalamic  reorganization  may  also 
involve  corticofugal  projections.  Krupa  et  al  have  shown  that  cortical  inactivation  also  significantly 
reduced  but  did  not  completely  eliminate  the  occurrence  of  VPM  receptive  field  reorganization  resulting 
from  the  reversible  peripheral  deafferentation  (PNAS,  1999). 

Conclusion 

These  results  of  this  study  provide  evidences  that  the  reversible  RF  changes  of  the  central  somatosensory 
system  after  TD  may  actually  involve  dynamic  alterations  of  the  global  as  well  as  local  neural  networks 
present  in  SI  cortex  and  VPL  thalamus. 
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Integration  of  Eye  Movements  and  Visual  Inputs 
at  the  Level  of  the  Lateral  Geniculate  Nucleus 

Joseph  Malpeli 

Department  of  Psychology,  University  of  Illinois 

Background 

For  higher  mammals,  the  density  of  cells  in  the  retina  decreases  dramatically  with  distance  from  the 
center  of  the  retina.  This  is  why  visual  acuity  is  highest  for  foveal  vision.  Consequently,  we  continually 
make  rapid  eye  movements  (called  saccades)  to  sample  different  regions  of  visual  space  with  this  narrow, 
high-resolution  window  on  the  world.  Saccadic  eye  are  particularly  interesting  because  they  reflect  the 
decision  to  attend  a  novel  object.  They  are  distinct  from  other  types  of  eye  movements,  both  in  their 
characteristics  and  in  the  brain  structures  that  have  evolved  to  generate  them.  For  example,  saccadic  eye 
movements  are  very  fast,  typically  reaching  several  hundred  degrees  per  second,  whereas  the  brain  systems 
for  following  moving  objects  with  smooth,  tracking  eye  movements  fail  at  speeds  greater  than  30-50 
degrees  per  second.  Saccades  are  too  fast  to  be  controlled  in  real  time  by  visual  feedback  -  once  initiated, 
they  are  terminated  on  target  by  internal  motor  circuits  that  estimate  the  distance  the  eye  traveled. 

The  Problem 

It  has  long  been  appreciated  that  the  strategy  of  viewing  the  world  through  fast,  frequent,  sequential 
samples  of  small  regions  of  visual  space  presents  several  challenges.  Because  of  their  high  speed,  saccades 
smear  the  image  on  the  retina.  However,  such  smearing  does  not  disturb  our  perception,  which  suggests 
that  there  are  active  mechanisms  in  the  brain  that  inhibit  visual  perception  during  saccades.  It  turns  out  that 
much  of  this  "saccadic  suppression"  results  because  retinal  ganglion  cells  don't  respond  well  when  images 
are  rapidly  swept  across  them  at  saccade  speeds  (i.e.,  "visual  masking"),  but  there  is  good  evidence  that 
active  central  suppression  has  a  role  as  well.  A  more  interesting  problem  is  that  images  acquired  over 
successive  saccades  must  be  integrated  into  a  unified  view  of  the  world.  We  often  make  several  saccades  a 
second,  so  clearly,  each  image  needs  to  be  evaluated  rapidly.  This  suggests  the  need  for  close  coordination 
of  perceptual  and  oculomotor  systems  at  early  levels  of  visual  processing. 

The  first  opportunity  for  interactions  between  visual  and  oculomotor  signals  is  in  the  lateral  geniculate 
nucleus  (LGN),  the  thalamic  structure  that  relays  information  from  the  eye  to  the  cerebral  cortex.  Axons 
of  retinal  ganglion  cells  terminate  on  the  relay  cells  making  up  the  LGN,  which  in  turn  relay  activity  to 
visual  cortex.  However,  of  the  several  million  axons  terminating  in  the  LGN,  only  about  10%  originate  in 
the  eye.  The  rest  come  from  several  areas  of  cerebral  cortex  and  a  variety  of  subcortical  structures,  some  of 
which  are  known  to  be  involved  in  the  generation  of  eye  movements.  In  this  talk,  I  will  describe  our  efforts 
to  reveal  effects  of  eye  movement  signals  on  the  responses  of  cells  in  the  LGN,  and  to  understand  their 
purpose.  I  will  argue  that  these  signals  likely  facilitate  the  processing  of  new  images  acquired  by  saccades, 
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and  are  not  responsible  for  suppressing  perception  during  saccades.  For  additional  information,  see  Lee,  D. 
and  J.G.  Malpeli.  J.  Neurophysiol.  79:  922-926,  1998. 

Methods 

These  experiments  were  performed  on  awake  cats,  trained  to  perform  an  oculomotor  task  for  food 
reward.  Eye  position  was  monitored  by  the  magnetic  search  coil  technique.  All  aspects  of  the  behavioral 
paradigm  and  data  collection  were  under  computer  control. 

The  cats  were  trained  to  fixate  a  small  laser  spot  presented  on  a  rear  projection  screen.  At  the  start  of 
each  trial,  a  laser  spot  came  on,  which  the  animal  had  to  quickly  fixate.  After  a  variable  delay,  the  first 
fixation  point  went  off,  and  simultaneously,  a  second  laser  spot  came  on  at  another  location,  to  which  the 
animal  made  a  saccade.  If  a  successful  saccade  was  made  from  the  initial  fixation  point  to  the  second  target, 
the  animal  was  rewarded  with  a  bolus  of  food.  Failure  to  acquire  targets  with  sufficient  speed  or  accuracy 
terminated  the  trial  without  reward.  In  recording  sessions,  the  activity  of  single  LGN  cells  was  monitored 
via  conventional  microelectrodes. 

During  various  phases  of  saccades,  a  large  (20  X  20  deg)  area  of  the  screen,  centered  on  the  receptive 
field  of  the  LGN  cell,  was  uniformly  dimmed  or  brightened  (depending  on  whether  the  LGN  cell  was 
excited  by  light  decrement  or  light  increment)  for  500  msec.  These  flashed  squares  were  dynamically 
centered  on  the  receptive  field  of  the  recorded  cell  during  the  saccade,  using  instantaneous  eye  position  to 
determine  the  location  of  the  stimulus.  They  were  never  related  to  the  reward  paradigm,  and  the  animal 
quickly  stopped  reacting  to  them  during  preliminary  behavioral  sessions.  Because  the  retinal  stimulus  did 
not  change  during  or  immediately  after  the  saccade,  any  variation  in  response  that  was  temporally  linked  to 
a  particular  phase  of  the  saccade  must  be  due  to  non-retinal  signals  synchronized  with  the  saccade. 

Activity  of  the  same  LGN  cells  was  also  recorded  for  spontaneous  saccades  made  in  total  darkness  to 
determine  if  any  saccade-related  modulations  of  activity  require  that  the  saccade  be  made  to  a  visual  target. 

Results 

LGN  activity  was  modulated  in  synchrony  with  the  saccade,  both  for  saccades  to  visual  targets  and  for 
saccades  in  the  dark.  To  quantify  these  effects,  we  compared  activity  in  brief  temporal  epochs  spanning  the 
saccade  with  activity  levels  during  a  baseline  period  well  after  completion  of  the  saccade.  Activity  levels  in 
the  dark  were  relatively  low,  typically  averaging  about  30  spikes/sec.  In  contrast,  the  response  to  the  flash 
was  complex  and  robust,  reaching,  on  average,  more  than  200  spikes/sec.  Nevertheless,  when  expressed  as 
a  fraction  of  the  post-saccadic  baseline,  activity  in  the  LGN  underwent  identical  changes  for  saccades  to 
visual  targets  and  saccades  in  the  dark.  This  means  that  over  a  very  wide  range  of  activity,  these  changes  in 
response  may  be  considered  multiplicative  variations  in  gain  -  that  is,  variations  in  the  ratio  of  output 
(LGN)  spikes  to  input  (retinal)  spikes.  Beginning  200-300  msec  prior  to  the  onset  of  the  saccade,  the  gain 
of  transmission  began  to  decrease,  reaching  a  modest  10%  drop  at  about  the  time  the  saccade  began. 
During  the  eye  movement,  gain  rapidly  increased,  reaching  and  surpassing  baseline  near  the  end  of  the 
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saccade,  peaking  at  about  a  20%  increase  in  gain  roughly  100  msec  after  the  eye  came  to  a  stop,  and 
gradually  declining  to  baseline  over  the  next  several  hundred  msec. 

These  gain  changes  were  qualitatively  similar  across  LGN  layer,  cell  class,  retinal  eccentricity,  and 
saccade  size.  However,  they  differed  greatly  for  the  two  major  classes  of  LGN  cells  -  X  and  Y  cells.  X 
cells  are  the  most  common  class  of  LGN  (and  retinal  ganglion)  cells.  They  form  a  high-resolution  system, 
and  they  give  a  relatively  sustained  response  to  a  maintained  stimulus.  Y  cells  are  less  common.  They  form 
a  high-sensitivity,  low-resolution  system,  and  they  give  transient  responses  to  maintained  stimuli.  The 
saccade-linked  changes  in  transmission  were  much  larger  for  X  cells  than  for  Y  cells. 

Conclusions 

Non-retinal,  saccade-related  signals  orchestrate  a  biphasic  change  in  the  gain  of  transmission  through 
the  LGN  that  is  time-locked  to  the  saccade.  This  change  does  not  seem  to  be  related  to  the  motivation  for 
making  the  saccade,  because  it  is  identical  for  rewarded  saccades  to  visual  stimuli  and  spontaneous 
saccades  made  in  darkness.  It  is  stable  over  activity  levels  varying  from  minimal  firing  rates  during 
darkness  to  vigorous  firing  rates  evoked  by  visual  stimuli.  The  initial  decrease  begins  so  early  that  it 
probably  reflects  the  decision  to  break  fixation  with  the  previous  target,  rather  than  the  subsequent  motor 
signal  initiating  the  saccade.  This  stereotyped  swing  from  a  modest  presaccadic  decrease  in  gain  to  a  larger 
postsaccadic  increase  maximizes  both  the  absolute  change  and  the  rate  of  change  of  visually-induced 
activity  associated  with  a  shift  of  gaze.  We  think  it  likely  that  this  speeds  up  the  activation  of  cortical  cells 
by  the  new  image.  Thus  we  propose  that  these  saccade-related  effects  on  the  LGN  function  to  enhance  the 
rapid  analysis  of  new  images  following  each  saccade.  The  observation  that  the  gain  swings  are  much  larger 
for  X  cells  than  for  Y  cells  adds  additional  support  to  this  notion.  Y  cells  respond  transiently  to  maintained 
images,  so  in  effect,  their  responsiveness  to  the  current  image  will  be  automatically  reduced  by  the  time  the 
next  saccade  is  launched,  and  there  is  less  need  for  an  active  gain  resetting  mechanism. 

These  saccade-related  signals  are  unlikely  to  have  much  to  do  with  perceptual  saccadic  suppression, 
because  the  timing  is  all  wrong.  The  suppression  begins  200-300  msec  before  the  eye  begins  to  move,  and 
it  begins  to  dissipate  while  the  saccade  is  at  peak  velocity.  Furthermore,  LGN  cells  respond  robustly  to 
visual  stimuli  during  all  phases  of  the  saccade.  If  their  response  produced  perceptual  confusion  because  of 
retinal  smearing  during  saccades,  it  is  difficult  to  see  how  this  modest  and  ill-timed  reduction  would 
alleviate  such  confusion. 
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Understanding  neural  mechanisms  by  which  voluntary  movements  are  produced  has  been  one  of  the 
key  issues  in  neuroscience.  Previous  studies  on  motor-related  cortical  areas  have  suggested  a  hierarchy 
among  them  in  terms  of  stages  of  motor  control.  The  primary  motor  area  (Ml)  appears  to  be  involved  in 
actual  execution  of  a  motor  program,  while  higher  motor  areas,  such  as  the  pre-motor  area  (PreMA)  and 
supplementary  motor  area  (SMA),  seem  involved  in  its  preparation.  Regarding  the  functional  role  of 
SMA,  however,  ideas  have  been  widely  varying.  Evidence  has  been  put  forth  that  suggests  SMA 
involvement  in  movement  selection,  preparation,  initiation,  execution,  and  feedback-monitoring  of  a 
motor  program  as  well  as  in  motor  learning  and  in  planning  of  complex  sequences  of  movement.  We  have 
conducted  a  series  of  experiments  using  the  high  temporo-spatial  resolution  of  functional  magnetic 
resonance  imaging  (fMRI)  to  study  the  relationship  between  activation  of  these  motor-related  cortical 
areas  and  various  stages  in  generation  and  comprehension  of  motor  programs  including  gestures. 

Experiment  1. 

The  first  study  aimed  at  examining  the  timing  of  activation  of  SMA  and  other  motor-related  areas 
with  respect  to  stages  of  movement  generation  using  a  delayed  motor  task  and  event-related  functional 
MRI.  Seven  normal  volunteers  performed  a  delayed-motor  task  in  which  the  preparation  of  finger 
movements  was  dissociated  in  time  from  movement  execution,  while  event-related  fMRI  was  obtained. 

The  Ml  and  PreMA  showed  expected  activation  associated  with  execution  and  preparation  stages, 
respectively.  Within  SMA,  sub-regions  with  different  temporal  profiles  of  activation  were  identified:  The 
anterior  part  became  activated  early  in  the  preparation  period  whereas  the  posterior  part  only  with 
movement  execution.  This  confirmed  in  human  brain  the  notion  that  the  classic  SMA  consists  of  the  pre- 
SMA  and  SMA  proper  each  with  different  functions. 

Experiment  2. 

Recently,  Rizzolatti  and  his  colleagues  suggested  that  primate  premotor  cortex  harbors  a  neural  system 
for  communication  using  gesture,  which  in  human  might  have  evolved  into  a  system  for  linguistic 
communication.  In  order  to  investigate  how  the  gestures  or  movements  of  other  conspecifics  are 
processed  in  human  brain,  event-related  functional  MRI  experiments  were  conducted  comparing  brain 
activation  associated  with  gesture  mimicking  and  gesture  comprehension  only  with  no  actual  movement 
made.  Novel  gestures  with  no  apparent  meaning  are  designed  to  keep  the  lingustic  component  of  the  task 
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to  a  minimum. 

The  SMA  and  premotor  area  of  both  hemispheres  as  well  as  visual  areas  in  the  occipital  lobe  and 
inferior  parietal  lobules  were  activated  in  association  with  presentation  of  the  novel  gestures  in  both 
gesture  mimicking  and  comprehension-only  tasks.  During  a  delay  period  when  the  mental  image  of  the 
gestures  were  supposedly  maintained  for  future  execution  in  mimicking  condition,  activation  of  premotor 
areas  extended  into  the  primary  motor  areas,  but  not  in  comprehension-only  task.  In  contrast,  left  inferior 
frontal  areas  or  Broca’s  area  was  highly  activated  only  in  comprehension  only  condition,  where  the 
subjects  tried  to  find  out  the  meaning  of  gestures,  despite  the  fact  that  the  gestures  were  novel  and 
deliberately  made  meaningless. 

These  findings  suggest,  first,  that  the  human  premotor  area  are  involved  in  a  system  by  which 
behaviors  of  conspecifics  are  perceived  and  mimicked  and  in  which  the  “mirror  neuron”  found  in  F5  of 
non-human  primate  brain  may  participate.  Second,  human  linguistic  system,  including  left  inferior  frontal 
cortex,  appears  to  be  separately  involved  in  the  processing  of  gesture  comprehension  only  when  semantic 
aspects  of  the  same  gestures  are  of  importance. 
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Mapping  and  Modeling  Human  Neural  Systems 
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The  purpose  of  this  lecture  is  to  demonstrate  that  non-invasive  brain-imaging  techniques  can  now  be 
used  to  create  comprehensive  maps  and  sophisticated  models  of  the  neural  systems  supporting 
perceptions,  actions,  cognitions,  and  emotions.  The  operational  definition  of  a  neural  system  is  the 
functional  brain  regions  and  inter-regional  connections  employed  in  the  performance  of  a  specific  task. 
The  majority  of  the  examples  are  drawn  from  studies  of  language.  In  particular,  speaking  a  written  word 
was  used  as  a  model  task  for  which  a  neural  system  map/model  was  desired. 

Neural  systems  mapping  and  modeling,  it  is  argued,  requires  data  of  three  types:  1)  operations  (task 
locations),  2)  connections,  3)  order.  An  operation  is  defined  as  the  information-processing  step 
performed  at  each  brain  region  activated  in  the  execution  of  the  task.  A  function  location  association 
must  be  defined  as  precisely  as  possible  in  terms  both  of  the  mental  operation  and  of  the  brain  location. 
A  connection  is  the  pathway  by  which  information  flows  from  one  activated  region  to  the  next.  Order  is 
the  sequence  and  timing  with  which  information  moves  from  one  location  to  the  next  in  the  performance 
of  a  task.  The  need  for  these  three  components  in  defining  a  model  of  a  neural  system  can  found  as  early 
as  the  work  of  Rene  Descartes  (1662)  and  has  been  a  consistent  feature  in  models  offered  by  researchers 
in  this  field  since  the  late  1800’s  and  to  the  present  day. 

Functional  brain  imaging  includes  a  range  of  techniques:  positron-emission  tomography  (PET), 
functional  magnetic  resonance  imaging  (fMRI),  event-related  electrical  potentials  (ERPs),  and 
magnetoencephalography  (MEG)  are  the  most  widely  used.  A  feature  common  to  all  functional  imaging 
techniques  is  the  ability  to  detect  task-induced  changes  in  brain  activity  either  directly  through  emitted 
electrical  and  magnetic  fields  (ERP,  MEG)  or  through  changes  in  hemodynamics  or  metabolism  triggered 
by  the  changes  in  neural  activity.  More  specifically,  these  techniques  are  used  to  detected  differences 
between  a  task  state  and  a  control  state:  here  termed  a  “conditional  contrast”  (CC). 

Conditional  contrasts  form  the  great  majority  of  functional  imaging  studies  to  date.  CC  studies,  for 
example,  have  been  used  extensively  to  map  the  functional  organization  of  language  comprehension  and 
production.  The  operational  maps  thus  derived  differ  in  many  important  ways  from  operational  maps 
derived  from  lesion-deficit  associations  (see  below).  The  most  initially  controversial  and  now  universally 
accepted  example  is  the  discovery  of  a  semantic  processing  zone  lying  in  front  of  Broca’s  area 
(Brodmann  area  46/47).  This  region  is  active  in  virtually  any  task  requiring  word  selection,  word 
completion  and,  to  a  lesser  degree,  word  comprehension.  Another  notable  example  is  the  very  recent 
acceptance  of  the  role  of  the  cerebellum  in  sensory  processing  and  attention;  that  is,  the  cerebellum  is  not 
solely  —  likely  not  even  primarily  —  a  motor  organ.  CC  studies  have  also  been  used  to  map  functions  for 
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which  lesion-deficit  studies  provided  little  or  no  information,  such  as  brain  organization  for  musical 
comprehension,  specific  emotions,  and  vegetative  appetites. 

Performance  correlation  is  a  newly  developed  strategy  for  functional  imaging  that  records  a  measure 
of  a  covert  (e.g.,  hallucinations)  or  overt  (e.g.,  spoken  syllables)  task  performance.  Image  data  are  then 
queried  for  regional  correlations  with  the  behavioral  measure.  This  strategy  is  invaluable  in  instances 
when  the  behavior  cannot  be  controlled  sufficiently  well  for  a  conditional  contrast.  It  is  also  valuable 
when  graded  performance  is  possible  or  specifically  desired. 

Connectivity  mapping,  unheard  of  even  a  few  years  ago,  is  now  in  rapid  evolution.  Effective 
connectivity  is  the  measurement  of  inter-regional  covariation  in  functional  data  during  the  performance 
of  a  task.  The  source  of  variance  from  which  the  inter-regional  connectivity  matrix  is  derived  is  of  two 
types:  inter-trial  variations;  intra-trial  variations.  Surprisingly,  it  appears  that  repetitive  performance  of  a 
brief  (e.g.,  1-2  second)  task  over  the  course  of  one  minute  is  associated  with  a  change  in  strategy  that  can 
be  used  to  segregate  sub-systems  from  one  another.  The  chief  disadvantage  of  this  method  is  that  it 
confounds  anatomical  connectivity  with  physiological  activations  in  a  manner  not  readily  deconvolved. 

TMS/PET  (transcranial  magnetic  stimulation  during  PET)  and  TMS/fMRI  are  very  recently 
developed  techniques  for  mapping  inter-regional  connectivity.  Focal  neuronal  depolarizations  are 
induced  immediately  under  the  center  of  an  B-shaped  TMS  coil.  This  activation  is  then  transmitted  trans- 
synaptically  to  remote  regions  connected  with  the  stimulated  site.  Both  the  local  and  remote  activations 
can  be  imaged  with  PET  and  fMRI.  Thus,  TMS  can  provide  a  non-invasive  means  of  imaging  and 
quantifying  inter-regional  connectivity. 

TMS  is  also  being  combined  with  imaging  as  a  means  of  making  a  bridge  between  traditional,  lesion- 
deficit  correlations  and  current  research  in  functional  brain  imaging.  It  has  long  been  known  that  brain 
imaging  identifies  areas  participating  in  a  task  but  does  not  distinguish  between  critical  areas  (those 
whose  absence  will  impair  task  performance)  and  non-critical  areas,  in  which  a  lesion  produces  no 
detectable  impairment  of  task  performance.  Lesion-deficit  analysis  identifies  only  “critical  areas”; 
functional  brain  imaging  identifies  both  critical  and  non-critical.  TMS,  when  used  in  combination  with 
functional  brain  imaging,  can  help  distinguish  critical  regions  from  non-critical. 

Order  (timing)  is  the  third  type  of  data  needed  for  mapping  and  modeling  human  neural  systems.  For 
mapping/modeling  task  performance  at  a  systems  level  (i.e.,  differences  in  timing  between  regions,  not 
between  processing  levels  within  a  region)  temporal  resolution  on  the  order  of  tens  to  hundreds  of 
milliseconds  is  required.  The  most  well-known  methods  for  achieving  high  temporal  resolution  in  brain 
imaging  are  electrical  (event-related  electrical  potentials  or  ERPs)  and  magnetic 
(magnetoencephalography  or  MEG).  Both  of  these  methods,  however,  are  hampered  by  the  “inverse 
problem”,  which  states  that  there  exists  no  unique  solution  for  identifying  the  brain  sources  of  externally 
detected  electrical  potentials  or  magnetic  fields.  In  short,  these  methods  have  poor  spatial  resolution  and 
are  fraught  with  spatial  ambiguities.  For  this  reason,  much  effort  has  been  placed  on  the  development  of 
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event-related  fMRI  methods.  Despite  the  slow  time  course  of  the  hemodynamic  response  to  neural 
activity  (used  by  both  fMRI  and  PET  as  an  index  of  brain  activity),  fMRI  can  be  used  to  distinguish 
differences  in  response  onset  as  fine  as  250  msec,  with  the  promise  of  even  higher  time  resolution  in  the 
near  future.  TMS  chronometry  -  in  which  the  TMS  pulse  is  presented  at  a  range  of  latencies  relative  to 
stimulus  delivery,  to  identify  the  precise  latency  of  processing  in  the  area  being  stimulated  —  is  a  non¬ 
imaging  technique  which  appears  capable  of  providing  processing  latency  information  precise  to  the 
order  of  tens  of  milliseconds. 

Collectively,  then,  functional  brain  imaging  and  adjunct  techniques  (such  as  TMS)  can  now  provide 
the  three  types  of  data  needed  for  mapping  and  modeling  human  neural  systems:  operations/locations, 
connections,  and  sequence. 
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Imaging  Brain  Function  and  Dysfunction 

Stephen  J.  Karlik 

Diagnostic  Radiology  and  Nuclear  Medicine,  University  of  Western  Ontario, 

Imaging  the  complex  functioning  of  the  brain  is  a  challenging  task.  Those  techniques  that  have  high 
temporal  resolution  may  have  inadequate  spatial  resolution.  Unfortunately,  the  opposite  is  also  true.  From 
the  point  of  view  of  the  neuroscientist,  the  ultimate  technique  for  imaging  function  would  identify  sites  of 
local  electrical  activity  throughout  the  brain  with  very  high  spatial  resolution  in  real  time.  This  goal  has  yet 
to  be  achieved. 

Functional  imaging  is  the  heart  of  the  newest  attempts  of  neuroscience  to  investigate  the  operation  of  the 
brain,  whether  applied  to  functional  MRI,  multi-channel  EEG,  magneto-encephalography  or  other 
techniques,  such  as  infared  imaging.  These  complimentary  techniques  can  be  used  to  establish  patterns  of 
activity  in  the  brain  which  can  lead  to  an  understanding  of  a  given  system  or  behavior.  The  purpose  of  this 
presentation  is  to  highlight  a  number  of  techniques  that  are  being  pursuing  in  the  quest  for  imaging  brain 
function.  These  examples  are  taken  from  my  own  work  and  that  of  other  members  of  the  Imaging  Research 
Laboratory  (www.irus.rri.on.ca). 

Localizing  functional  activity:  Current  techniques  for  deep  brain  microsurgery  require  identification  of 
targets  by  preoperative  imaging  localization.  Many  critical  structures  targeted  in  this  way  (the  thalamic 
nuclei)  are  functionally  distinct  but  not  discemable  on  MR  images.  Critical  brain  areas  that  must  not  be 
damaged  by  the  surgical  procedure  also  surround  these  structures;  these  factors  make  accurate  localization 
of  lesion  targets  crucial.  Digitized  anatomical  atlases  derived  from  histological  stained  brain  specimens 
registered  to  patient  MRI  data  sets  aid  in  delineating  targets,  but  the  accuracy  of  registration  within 
homogeneous  anatomy  remains  questionable.  To  illustrate  this  problem,  a  searchable  and  expandable 
database  of  functional  organization  for  the  sensorimotor  thalamus,  internal  capsule,  and  internal  pallidum 
has  been  designed  from  a  population  of  patients  (n=52).  Data  were  obtained  through  microcellular 
recording,  microstimulation,  and  macrostimulation  mapping  performed  during  stereotactic  thalamotomies 
and  pallidotomies.  After  registration  of  the  database  into  standard  stereotactic  space,  clustering  of  like- 
physiological  responses  was  noted  in  the  internal  capsule  and  sensorimotor  thalamus  and  an  articulated 
joint-based  organization  were  observed  in  the  internal  pallidum.  Furthermore,  a  clear  delineation  of  the 
kinesthetic-paresthetic  functional  border  was  observed  within  the  thalamus.  When  registered  to  a  patient 
MRI  within  an  image-guided  visualization  platform,  this  type  of  database  provides  a  visual  representation 
of  deep  brain  functional  organization  facilitating  physiological  exploration  of  normal  functional  anatomy 
and  permitting  preoperative  planning.  This  portion  of  the  presentation  is  thus  focused  on  the  techniques 
needed  to  coregister  electrophysiological  and  conventional  3D  MRI  data  in  a  large  patient  population. 

JMRI  at  high  resolution  spatial  and  temporal  resolution:  Advances  in  single-event  functional  magnetic 
resonance  imaging  (MRI)  have  allowed  the  extraction  of  relative  timing  information  between  the  onset  of 
activity  in  different  neural  substrates  as  well  as  the  duration  of  brain  processing  during  a  task,  offering  new 
opportunities  in  the  study  of  human  brain  function.  Single-event  fMRI  studies  have  also  facilitated 
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increased  spatial  resolution  in  fMRI,  allowing  studies  of  columnar  organization  in  the  human  cortex.  Since 
important  processes  such  as  object  recognition,  binocular  vision  and  other  processes  are  thought  to  be 
organized  at  the  columnar  level,  these  advances  in  the  spatial  and  temporal  capabilities  of  fMRI  allow  a 
new  generation  of  basic  neuroscience  studies  to  be  performed,  investigating  the  temporal  and  spatial 
relationships  between  these  cortical  sub-units.  Such  experiments  bear  a  closer  resemblance  to  single-unit 
or  evoked-potential  studies  than  to  classical  static  brain  activation  maps  and  might  serve  as  a  bridge 
between  primate  electrophysiology  and  human  subjects.  This  section  of  the  talk  will  describe  specific 
examples  of  the  latest  advances  from  our  4T  fMRI  unit.  Some  of  the  research  topics  are:  object  versus  facial 
identification  in  the  fusiform  gyrus,  motor  learning  tasks,  understanding  saccades  and  anti-saccades, 
analysis  of  ocular  dominance  in  amblyopia,  and  examining  the  motor-sensitive  MT  /V5  cortex. 

fMRI  of  the  visual  system  in  multiple  sclerosis:  We  are  performing  experiments  to  examine  the 
functional  MRI  (fMRI)  response  in  MS  patients  with  optic  neuritis  -  an  impairment  of  the  optic  nerve.  We 
compared  neuronal  activation  measured  with  fMRI  to  the  delay  in  visual  evoked  potentials  (VEPs  -  a 
common  clinical  measure  of  optic  nerve  damage)  and  found  significant  correlation.  Thus,  MRI  may  be 
capable  of  providing  not  only  an  anatomical  indication  of  disease 

presence  and  evolution,  but  also  a  functional  indication  as  it  correlated  with  an  independent  measure  of 
brain  function  measured  by  electrical  activity.  This  section  of  the  presentation  will  examine  the  novel 
approach  of  understanding  altered  cortical  evoked  potentials  in  reference  to  underlying  physiological 
deficits  measurable  with  fMRI  techniques. 

Brain  circulatory  parameters  measured  with  CT:  Although  fMRI  typically  represents  the  new  modality 
of  choice  for  the  examination  of  functional  activity  of  the  brain,  contrast-enhanced  CT  can  provide  a 
valuable  look  at  hemodynamic  properties,  especially  as  a  response  to  an  insult.  A  technique  has  been 
developed  to  acquire  and  model  CT  data  to  yield  images  of  blood  volume  and  blood  flow  in  the  brain.  The 
important  responses  of  the  vasculature  to  a  tumor  or  stoke  can  be  visualized  and  quantified.  Such 
measurements  are  currently  unavailable  for  MRI  and  serve  to  illustrate  that  advanced  image  processing 
techniques  can  extract  new  and  important  information  from  an  established  technology. 

Multispectral  analysis:  MRI  can  provide  multiple  images  of  the  same  anatomy,  each  with  different 
contrast  characteristics.  Contrast  is  controlled  by  selection  of  imaging  pulse  sequence  parameters.  Images 
can  be  produced  with  contrast  weighted  by  a  variety  of  factors  including  proton  density  (PDw),  T1 
relaxation  time  (Tlw),  T2  relaxation  time  (T2w),  diffusion  (Dw)  and  magnetization  transfer  (MTw). 
Multi-contrast  MR  images  are  also  referred  to  as  "multi-spectral"  images  -  each  contrast  is  treated  as  a 
component,  or  "band”  making  up  an  "MR  spectrum".  Typically,  multi-spectral  MR  images  are  acquired 
such  that  tissues-of-interest  have  different  contrast  characteristics  in  one  or  more  of  the  spectral  bands. 
Analysis  of  multi-spectral  MR  data  is  one  of  the  most  exciting  and  promising  areas  of  medical  image 
processing.  This  technology  is  based  on  remote  sensing  software  developed  for  the  analysis  of  satellite  data. 
Consequently,  multi-spectral  images  can  provide  much  more  information  than  analysis  of  a  single  contrast 
image  and  in  so  doing  provide  an  opportunity  to  extract  additional  new  information.  The  use  of 
multispectral  analysis  on  medical  images  is  a  unique  idea.  This  portion  of  the  talk  will  examine  the 
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computer  programs  used  and  the  results  that  can  be  obtained  from  multiple  data  sets. 

There  are  a  number  of  very  interesting  other  imaging  modalities  that  should  be  also  considered. 
Included  among  these  are:  magnetoencephalography,  infared  imaging,  multi  channel  (256  channel)  EEG, 
positron  emission  tomography  and  single  photon  emission  tomography.  Key  to  employing  any  of  these 
techniques  is  not  only  a  detailed  knowledge  of  the  technique  itself,  but  an  essential  appreciation  of  their 
strengths  and  weaknesses  to  enable  the  investigator  to  design  the  appropriate  study.  This  even  holds  for 
established  techniques,  such  as  fMRI,  where  today  the  experimental  paradigm  is  far  more  important  than 
the  imaging  technique  itself.  However  difficult  the  inherent  challenges  of  an  imaging  modality,  each  type 
of  imaging  can  provide  additional  new  information  on  brain  function  when  the  best  experimental  paradigms 
are  selected. 

One  critical  area  for  future  research  will  be  in  the  general  area  of  molecular  imaging.  Many  imaging 
modalities  have  had  a  substantial  increase  in  image  resolution  due  to  technological  improvements  over  the 
past  10  years.  We  have  microCT  and  MRI  facilities  that  have  microscopic  imaging  abilities.  This 
equipment  allows  the  non-destructive  examination  of  tissues  or  animals  with  the  eventual  goal  being 
histopathological  analysis.  Molecular  imaging  takes  this  idea  another  step  forward  in  the  assessment  of 
function  and  can  do  so  in  a  three-dimensional  way,  to  replace  time-consuming  and  labor-intensive  methods 
used  at  present.  The  techniques  are  based  on  the  ability  of  the  imaging  system  to  identify  molecules  rather 
than  structures.  Some  examples  are  identifying  labeled  macrophages  invading  a  tissue  using  MRI,  tracking 
gene  therapy  using  scintigraphy,  or  optical  imaging  of  enzyme  activity.  It  is  essential  to  consider  this  line 
of  investigation  when  imaging  of  function  and  dysfunction  of  the  brain. 
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Prefrontal  activations  during  different  memory  processings  : 

fMRI  studies 
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Functional  Neuroanatomy  has  been  studied  using  fMRI  method  during  repetition  priming  (implicit 
memory)  and  recognition  (explicit  memory)  tasks.  Material-specific  effects  were  examined  by 
comparing  brain  activations  associated  with  encoding  and  retrieval  of  picture  and  word  stimuli.  To 
investigate  material  specific  effects  on  priming  (Studyl),  picture  or  word  stimuli  were  presented  while 
participants  performed  a  semantic  classification  task  (man-made/natural).  Initial  performance  and 
following  repeated  performance  of  the  same  classification  task  were  compared  for  picture  and  word 
each.  Reduction  of  brain  activation  was  found  during  the  repeated  performance  in  brain  regions 
that  were  involved  in  initial  encoding  processing.  Depending  on  perceptual  format  of  stimuli,  the 
brain  activation  reduction  was  found  mainly  in  left  inferior  prefrontal  region  for  word  while  it  was  in 
bilateral  occipital  cortex  for  picture.  Change  of  perceptual  format  between  encoding  and  retrieval 
episodes  also  studied  by  comparing  word-studied-picture-tested  condition  (WP  condition)  and 
picture-studied-picture-tested  condition  (PP  condition).  In  WP  priming  condition,  right  prefrontal 
activation  was  observed,  relative  to  PP  priming  condition,  in  addition  to  bilateral  occipital  activation. 
Explicit  retrieval  (recognition)  was  also  examined  (Study  II)  both  during  PP  recognition  condition 
and  WP  recognition  condition.  In  PP  recognition  condition,  increases  of  brain  activations  were 
found  in  bilateral  prefrontal  cortexes,  parietal  cortex,  and  occipital  cortex,  relative  to  fixation 
baseline.  In  WP  recognition  condition,  however,  increased  activation  was  found  in  left  prefrontal 
region  only.  These  findings  suggested  that  not  only  the  type  of  memory  but  also  perceptual  format  of 
stimulus  between  encoding  and  retrieval  episode  should  be  considered  to  understand  functional  role 
of  prefrontal  cortex  in  memory. 


INTRODUCTION 

The  idea  that  memory  involves  multiple  processings  and  systems,  has  been  supported  by  a  number  of 
researches,  both  with  human  and  animal  models.  Functional  neuroimaging  studies  have  shown  brain 
activation  in  various  brain  regions  associated  with  different  memory  processing.  Previous  experience  of 
episodes  or  events  affects  behavior,  either  with  or  without  intentional  or  conscious  recollection  of  the 
previous  episodes.  Retrieval  of  previous  episodes  with  conscious  recollection  is  called  explicit  retrieval 
whereas  retrieval  without  conscious  recollection  is  called  implicit  retrieval.  Repeated  processing  of 
previous  events  often  resulted  in  fast  and  accurate  behavioral  responses  in  later  processing.  This 
phenomenon  is  referred  as  repetition  priming,  considered  as  retrieval  of  implicit  memory.  It  has  been 
often  observed  with  reduction  of  brain  activations,  during  repeated  processing,  in  areas  involved  in  initial 
processing.  In  contrast  to  repetition  priming,  increases  of  brain  activations  were  also  observed  during 
conscious  recollection  of  previous  episode,  considered  as  retrieval  of  explicit  memory.  A  number  of 
studies  indicated  that  prefrontal  cortex  (mainly  right  side)  was  involved  in  explicit  retrieval,  whereas  the 
left  prefrontal  cortex  is  involved  in  encoding.  However,  the  precise  role  of  right  prefrontal  and  left 
prefrontal  regions  during  episodic  retrieval  have  not  yet  been  settled  among  researchers3,4.  Considering 
prefrontal  cortex  is  not  the  only  candidates  of  neural  substrates  of  memory5,  further  investigations  are 
needed  to  understand  the  nature  of  memory  processing  and  its  functional  neuroanatomy. 

While  debating  asymmetry  among  left  and  right  brain  regions  in  retrieval  processing,  researchers  have 
often  ignored  the  way  in  which  material  type  of  encoded  items  during  study  episodes  interacts  with 
retrieval  processing,  which  also  depends  on  the  perceptual  format,  i.e.,  material  type,  of  retrieval  cues. 
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This  might  be  the  case  whether  it  is  explicit  retrieval  or  implicit  retrieval  processing.  In  literatures,  it 
has  been  well  documented  that  encoding  involves  both  perceptual  processing  and  conceptual  processing, 
especially  during  a  semantic  task.  Therefore,  different  perceptual  formats  might  elicit  different 
perceptual  processing,  which  in  turn,  result  in  different  brain  activations  both  during  encoding  and 
retrieval.  If  that  is  the  case,  it  is  very  likely  that  perceptual  format  of  encoding  interacts  with  that  of 
retrieval  and  the  brain  activation  patterns  as  well.  If  material  types  (i.e.,  word,  picture,  or  face)  or 
stimulus  modalities  (visual,  auditory)  of  retrieval  cue  are  consistent  with  the  format  in  which  the  target 
items  have  been  encoded,  it  will  affect  the  perceptual  processing  of  retrieval.  In  explicit  retrieval,  this 
consistency  might  be  a  basis  of  the  perceptual  familiarity  which  needs  to  be  evaluated,  or  utilized  during 
the  recollection  process.  In  implicit  retrieval,  this  consistency  might  be  a  basis  of  facilitation  in 
perceptual  processing  during  repeated  processing.  In  other  words,  the  former  requires  further  processing 
of  evaluation  whereas  the  letter  requires  less  processing  requirement.  These  differences  in  retrieval 
processing  might  be  associated  with  increase  of  brain  activation  (explicit  memory)  or  decrease  of  brain 
activation  (implicit  memory).  Still,  it  has  not  been  well  defined  which  retrieval  process  might  occur 
when  physical  format  of  encoded  stimulus  was  changed  at  the  time  of  retrieval.  Furthermore,  it  has 
seldom  been  studied  which  brain  areas  are  involved  in  this  cross  format  retrieval  processing,  either  in 
implicit  memory  task,  such  as  repetition  priming,  or  explicit  memory  task,  such  as  recognition. 

In  the  following  two  studies,  it  was  investigated  the  brain  regions  that  activated,  or  deactivated  in 
retrieval/encoding  processing,  depending  on  the  perceptual  format  of  stimulus  at  the  time  of  encoding  and 
retrieval.  This  type  of  studies  will  provide  further  insights  in  various  information  processing  components 
of  retrieval. 


Study  I. 


Priming  and  Reduction  of  Brain  Activation 

It  has  well  been  known  that  speed  and  accuracy  increases  when  same  events  or  episodes  are  processed 
repeatedly,  relative  to  initial  processing.  Evidences  suggested  that  repeated  processing  of  identical 
events  is  associated  with  reduction  of  brain  activation  of  the  brain  areas  involved  in  initial  processing  of 
the  episodes1,2  Modality  change  between  study  and  test  seemed  to  affect  both  implicit  and  explicit 
memory,  but  more  in  implicit  memory,  probably  due  to  reduction  of  perceptual  priming.  For  example, 
words  studied  as  visual  modality,  were  better  recognized  or  primed  with  visual  words  than  auditory  word, 
vice  versa.  Even  though  this  implies  influence  of  past  experience  on  perceptual  processing,  priming 
effect  was  reported  to  still  exist  in  cross-modality  condition,  reflecting  contribution  of  semantic  priming 
or  semantic  retrieval.  The  studies,  reported  here,  were  performed  in  the  interest  of  two  questions.  The 
first  was  to  investigate  material  type  effects  on  semantic  repetition  priming  and  its  brain  activation 
patterns,  while  the  second  was  to  dissociate  brain  regions  involved  in  perceptual  processing  and 
conceptual  processing  of  semantic  task.  Material  specific  effect  was  studied  with  two  different  kinds  of 
visually  presented  stimulus  type,  i.e.,  word  (WW  scan)  and  picture  (PP  scan).  Brain  activations  during 
initial  presentation  and  repeated  presentation  were  compared  to  observe  the  priming  effects  for  each 
stimulus  types  (Study  I).  To  identify  the  brain  areas  involved  in  semantic  processing,  priming  condition 
was  designed  to  contain  different  perceptual  processing,  but  repeated  semantic  processing.  This 
particular  priming  condition,  called  a  cross  priming  condition,  was  compared  with  a  regular  priming 
condition  in  a  scans.  In  other  words,  the  perceptual  feature  of  previous  episode  was  changed  during 
retrieval  in  cross  priming  condition  (word-studied-picture-tested:  WP  condition).  This  was  compared 
with  a  within  priming  condition  (picture-studied-picture-tested:  PP  condition)  in  the  third  scan  (CROSS 
scan).  The  WP  priming  condition  and  PP  priming  condition  shared  same  amount  of  semantic  priming 
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but  only  the  PP  priming  condition  had  both  conceptual  priming  and  perceptual  priming.  Therefore,  the 
difference,  if  there  was  any,  between  cross  priming  condition  and  within  priming  condition  must  reflect 
difference  in  perceptual  priming,  not  conceptual  priming.  The  difference  observed  between  WP  priming 
and  PP  priming  could  be  related  with  difference  in  perceptual  priming.  Any  observed  difference  in  brain 
activations  between  two  conditions  in  CROSS  scan,  therefore,  must  be  associated  with  perceptual 
processing  specific  to  the  stimulus  format. 

Methods  and  Procedures:  Scan  was  performed  in  a  1 .5  T  GE  MR  machine  with  T2*-sensitive  gradient 
echo  spiral  pulse  sequence(TE  =  40  ms,  TR  =  1440  ms,  flip  angle  =  80°,  FOV  =  24  cm  and  inplane 
resolution  =  1.875  mm,  6mm  thick  16  slices,  two  spiral  interleaves).  Healthy  participants  (n=9)  were 
scanned  during  three  scans  while  semantic  classification  tasks  were  performed  for  pictures  (PP  scan: 
initial  picture  vs.  repeated  picture;  CROSS  scan:  WP  priming  vs.  PP  priming),  word  (WW  scan:  initial 
word  vs.  repeated  word),  WP  scan  (WP  scan:  word-to-picture  vs.  picture-to-picture).  Participants  viewed 
the  visually  presented  line  drawing  image  or  word  (100ms  duration/2sec)  from  a  mirror  mounted  above 
the  subject’s  head,  which  reflected  the  image  from  the  screen.  Reaction  time  was  recorded  by  a 
computer  outside  of  the  scanner.  Encoding  was  achieved  while  semantic  classification  judgment  task 
(man-made/natural)  was  performed  both  for  picture  stimulus  list  and  for  word  stimulus  list.  In  WW 
scan  and  PP  scan,  a  scan  were  composed  of  eight  cycles  of  alternating  initial  presentation  condition  (15 
trial/half-cycle)  and  repeated  representation  condition.  WP  scan  was  consisted  of  eight  cycles  of  WP 
priming  condition  (word-studied-picture  -ested)  and  PP  priming  condition  (picture-studied-picture-tested). 

Behavioral  Results:  The  performance  of  semantic  classification  task  was  measured  in  accuracy  and 
reaction  time.  There  was  significant  reduction  in  reaction  time  (PP:  p  <.001,  WW:  p  <0.005)  and 
accuracy  (picture:  p  <.001 )  during  repeated  cycles,  in  comparison  to  the  initial  cycles  during  the  first  two 
scans.  WP  priming  condition  was  associated  with  longer  latency  (p  <.05)  and  lower  accuracy  (p  <.01 )  in 
comparison  to  PP  priming  condition  of  CROSS  scan. 

Brain  Activation:  Reduction  of  brain  activation  was  found  during  semantic  classification  processing  of 
the  repeated  picture,  relative  to  initial  presentation.  Significant  reduction  with  repeated  picture  (PP 
scan)  was  found  in  occipital  cortex  bilaterally,  such  as  left  fusiform  gyrus,  right  middle  occipital  gyrus, 
and  bilateral  cuneous.  With  repeated  word  presentation  (WW  scan),  reduction  of  brain  activation  was 
also  found  in  left  inferior  prefrontal  cortex  (BA  44),  medial  superior  prefrontal  (BA  3/32),  and  bilateral 
cuneous.  In  CROSS  scan,  WP  priming  (word-studied-picture-tested  condition),  relative  to  PP  priming 
(picture-studied-picture-tested  condition),  resulted  in  greater  activation  in  various  brain  regions,  such  as 
left  fusiform  gyrus,  left  tempo-parietal-occipital  region,  and  right  middle  occipital  region.  A  greater 
activation  was  also  observed  in  posterior  region  of  the  right  middle  prefrontal  cortex  (BA  6)  during  the 
WP  priming  condition,  in  comparison  to  PP  priming  condition. 

Study  II. 

Explicit  Retrieval  and  Brain  Activation 

Explicit  retrieval  processing  was  investigated  both  in  PP  condition  and  WP  condition  respectively. 
Recognition  task  was  given  following  incidental  encoding  of  picture  items  and  word  items.  In  PP 
recognition  condition,  the  previously  studied  picture  items  were  given  as  target  items  along  with 
distracters.  In  WP  recognition  condition,  the  studied  word  items  were  presented  as  picture  form  as 
target  items,  along  with  picture  distracters.  Therefore,  the  target  items  of  PP  condition  provided  both 
perceptual  and  conceptual  familiarity,  whereas  the  target  times  of  WP  condition  did  conceptual  familiarity, 
but  not  perceptual  familiarity.  Recognition  occurred  based  on  these  familiarities  coming  from  old 
stimuli,  in  comparison  to  new  items.  Therefore,  it  was  expected  that  PP  recognition  condition  might 
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have  been  easier  for  recognition  task  than  the  WP  condition.  Greater  activation  of  brain  of  PP  condition, 
relative  to  WP  condition,  could  have  reflected  the  retrieval  processing  based  on  perceptual  familiarity. 

Methods  and  Procedures:  Scan  was  performed  in  a  1.5  T  GE  MR  machine  with  EPI  capability. 
Healthy  participants  (n=16)  were  scanned  during  picture/word  semantic  classification  tasks  (two  study 
scans),  and  followed  by  two  memory  tasks  (two  test  scans).  Encoding  was  achieved  while  semantic 
classification  judgment  task  (man-made/natural)  was  performed  both  for  picture  stimulus  list  and  for 
word  stimulus  list.  Each  test  scan  (PP  scan,  WP  scan)  was  composed  of  five  alternating  cycles  of 
recognition  condition  and  repeated  encoding  condition,  even  though  only  the  results  of  explicit  memory 
condition  were  reported  here.  During  the  recognition  task  cycles,  memory  was  tested  with  all  picture 
form.  Therefore,  in  recognition  condition  cycle  of  PP  scan,  the  target  items  came  from  the  picture  study 
list  and  tested  with  picture  stimuli,  while,  in  WP  scan,  they  did  from  the  word  study  list  and  tested  with 
picture  stimuli.  A  rest  period  with  a  cross  hair  fixation  point  was  given  beginning  and  end  of  all  the 
scans,  serving  as  common  baseline  for  both  scans. 

Behavioral  Results:  The  performance  of  recognition  task  was  measured  from  eleven  participants 
during  scanning  and  the  behavioral  data  were  subject  to  further  analysis.  Level  of  recognition 
performance  differed  between  PP  condition  and  WP  condition.  The  recognition  performance  of  PP 
condition  was  better  in  term  of  reaction  time  (p  <.0001)  and  correct  response  rate  (p  <.0.005)  but  not  in 
error  rate  (p  >.73). 

Brain  Activation:  Brain  activation  was  found  in  occipital  cortex  and  posterior  parietal  cortex 
bilaterally  during  both  recognition  conditions,  relative  to  baseline  condition.  In  PP  condition  that  was  a 
typical  recognition  condition,  brain  activations  were  also  found  in  left  inferior  prefrontal  cortex,  medial 
superior  prefrontal  cortex,  and  right  prefrontal  cortex.  Unlike  PP  condition,  no  right  prefrontal 
activation  was  found  during  WP  recognition  conditions  in  which  no  perceptual  familiarity  was  given  by 
target  items  (See  Figure  1).  This  suggested  that  the  right  prefrontal  activation  may  not  be  necessarily 
associated  with  retrieval  process  itself,  but  may  be  associated  with  evaluation  of  perceptual  familiarity 
during  retrieval.  These  findings  also  suggest  that  left  prefrontal  area  might  involve  in  explicit  retrieval 
of  semantic  aspects  of  previous  episodes.  No  additional  brain  areas  were  recruited  in  order  to 
compensate  changes  in  physical  format  of  stimulus  between  encoding  and  retrieval. 

Recognition  Test  with  Picture  Stimulus  Form 


PP  condition 

(picture-studied-picture-tested) 


WP  condition 

(word-studied-picture-tested) 


Figure  1:  Brain  activation  during  PP  and  WP  recognition  tasks,  relative  to  fixation.  Prefrontal 
activation  was  observed  bilaterally,  relative  to  simple  fixation  condition,  during  the  PP  recognition  task 
but  only  in  the  left  side  during  the  WP  recognition  condition. 
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DISCUSSION 


Implicit  memory  and  explicit  memory  were  studied  in  within-  and  cross-  stimulus  form  conditions, 
using  repetition  priming  (Study  I)  and  recognition  task  (Study  II).  In  both  studies,  pictures  and  word 
stimuli  were  incidentally  encoded  during  a  semantic  classification  tasks.  In  repetition  priming  study, 
during  the  later  performance,  relative  to  initial  performance,  different  stimulus  types  resulted  in  reduction 
of  brain  activation  in  different  brain  regions.  Recognition  task,  however,  resulted  in  increase  of  brain 
activation  in  various  brain  regions,  relative  to  encoding  task,  or  simple  baseline  fixation  (Study  II). 
These  findings  indicated  that  brain  activation/deactivation  patterns  differed  depending  on  memory  task 
types  and  stimulus  types. 

The  reduction  of  activation  due  to  priming  (implicit  memory)  during  semantic  classification  task  was 
consistent  with  previous  findings  in  other  studies  of  repetition  priming1, 6.  First,  material  specific  effect 
of  implicit  memory  was  observed  during  repetition  priming.  The  brain  regions  showing  brain 
deactivation  were  mainly  in  left  prefrontal  activation  for  word  stimulus  but  bilateral  occipital  cortex  for 
picture  stimulus.  This  material-specific  difference  might  suggest  difference  in  degree  to  which 
conceptual  processing  and  perceptual  processing  were  involved  in  semantic  classification  task,  even 
though  the  task  requirement  was  identical.  Second,  WP  priming,  in  comparison  to  PP  priming,  resulted 
in  activations  in  occipital  cortex  but  not  in  left  prefrontal  cortex,  indicating  need  of  initial  stimulus- 
specific  perceptual  processing  but  not  conceptual  processing. 

Increase  of  brain  activation  (Explicit  memory)  was  found  during  recognition  task.  First,  right 
prefrontal  activation  was  found  in  relation  with  explicit  retrieval  of  PP  recognition  condition,  as  well  as 
left  prefrontal  activation.  However,  the  right  prefrontal  activation,  which  has  been  suggested  as  brain 
activation  associated  with  explicit  retrieval,  was  found  in  PP  recognition  condition  but  not  in  WP 
recognition  condition.  Right  prefrontal  cortex  in  this  explicit  retrieval  task  may  have  been  associated 
with  perceptual  familiarity  monitoring,  not  necessarily  retrieval  processing  per  se.  Second,  inferior  left 
prefrontal  activation  was  found  in  PP  recognition  condition  while  middle  prefrontal  activation  was  found 
in  WP  recognition  condition.  The  difficulty  of  performance  in  WP  recognition  condition,  relative  to  PP 
recognition  condition,  might  explain  the  left  middle  prefrontal  cortical  activation.  Lower  recognition 
performance  of  WP  recognition  condition  (lower  accuracy,  greater  latency)  indicates  a  possibility  that 
participants  might  have  been  involved  in  strategic  evaluation  during  retrieval  process. 

The  WP  (word-studied-picture-tested)  conditions,  both  in  repetition  priming  and  recognition  tasks 
resulted  in  different  activation  patterns.  The  function  of  right  prefrontal  activation  observed  during  WP 
priming  condition  is  difficult  to  understand,  but  still  very  intriguing.  Usually,  right  prefrontal  cortical 
activation  could  be  attributed  to  explicit  retrieval  during  a  nominal  implicit  memory  task,  even  though  it 
was  not  intended  in  Study  I.  However,  it  is  unlikely  the  case  since  the  explicit  retrieval  of  WP  condition 
in  the  following  recognition  study  (Study  II)  was  associated  exactly  with  left  prefrontal  cortex,  not  with 
the  right  prefrontal  cortex.  According  to  a  follow-up  priming  study 7 ,  the  right  prefrontal  activation  was 
observed  again  in  WP  condition,  but  not  in  PW  condition  (picture-studied-word-tested  condition).  The 
left  prefrontal  activation  observed  in  explicit  retrieval  task  was  often  attributed  to  additional  encoding  of 
distracters  or  re-encoding  of  weekly-encoded  target  materials,  instead  of  retrieval  requirements. 
According  to  the  results  of  this  study,  it  was  less  likely  since  new  materials  were  pictures  whose  encoding 
was  more  likely  bilateral,  instead  of  left  only.  Also  the  precise  location  of  left  prefrontal  activations  in 
the  WP  recognition  between  the  brain  activation  during  new  picture  encoding  differed.  Several 
possibilities  can  be  considered.  The  left  prefrontal  activation  in  WP  condition  might  be  understood  as 
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strategic  processing  of  WP  recognition  condition,  or  related  with  the  fact  that  the  target  item  was 
originally  encoded  as  word  form,  whose  activation  was  usually  associated  with  left  prefrontal  activation. 
These  possibilities  remained  to  be  further  explored  in  future  studies.  These  interesting  contrasts 
between  implicit  memory  task  and  explicit  memory  task  need  further  investigation  in  near  future. 
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Agrammatism  in  Broca's  Aphasia: 
Dissociation  of  Sentence  Production  and  Comprehension 


Kichun  Nam 

Department  of  Psychology,  Korea  University 

In  the  current  study,  the  syntactic  processing  related  with  the  comprehension  and  production  processes 
was  examined.  In  order  to  see  whether  the  agrammatic  Broca  aphasic  has  difficulty  in  comprehending  and 
producing  the  subjective  and  objective  cases  and  these  processes  are  modular,  the  sentence-picture 
matching  task  and  two  sentence  completion  tasks  were  employed. 

The  sentence-picture  matching  task  was  to  evaluate  the  comprehension  performance  of  the 
semantically  reversible  sentences.  The  semantically  reversible  sentences  can't  be  understood  correctly 
without  understanding  the  subjective  and  objective  case  markers.  For  example,  in  the  sentence-picture 
matching  task,  7j-  •#<>)#  •§-£}-”  (in  English,  the  sentence  means  that  "a  crocodile  bites  a  shark") 
was  presented  to  the  Broca  aphasic,  and  then  the  Broca  aphasic  was  asked  to  choose  a  correct  picture  to 
describe  the  sentence.  As  shown  in  table  1 ,  the  result  of  the  sentence-picture  matching  task  showed  that 
the  agrammatic  Broca  patient  comprehended  the  semantically  reversible  sentences  almost  100%  correctly. 
The  incorrect  responses(4  out  of  32)  can't  be  interpreted  as  real  errors,  because  4  pairs  of  pictures 
describing  the  sentences  were  ambiguous.  Thus,  the  patient  comprehends  the  subject  and  object  case 
markers  almost  perfectly.  Because  the  semantically  reversible  sentences  can’t  be  comprehended  without 
understanding  the  case  markers  correctly,  this  result  suggests  that  he  can  use  the  case  markers  in 
comprehension  successfully. 

Table  1 .  Performance  in  the  sentence-picture  matching  task 


Correct 

Incorrect 

Total 

Response 

Responses 

87.5% 

12.5% 

100% 

(28/32) 

(4/32) 

(32  cases) 

Two  sentence  completion  tasks  were  used  to  evaluate  the  production  processing  of  the  case  markers 
and  morphological  inflections:  one  was  to  select  the  correct  case  markers  among  several  alternative  case 
markers  to  fill  in  the  empty  blanks  involved  in  the  test  sentence(hereafter,  it  is  named  "the  case  selection 
task),  and  the  other  was  to  generate  an  appropriate  case  markers  without  any  alternative 
presentations(hereafter,  it  is  named  "the  case  generation  task).  One  example  sentence  presented  in  the 
case  selection  task  is  as  follows. 

g7T-¥j  h  is vr  Irl  k  ^t^s]-^cj-(in  English,  which  means  that  "my  mother 
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mixed  flour  with  boiling  water"). 

The  correct  answers  for  the  1st,  2nd,  and  3rd  blanks  are  “7]-”  (subject  case  marker),  (object  case 
marker)  and  "  °i]  ”  (place  case  marker?)  respectively.  In  this  task,  the  Broca  patient  selected  around  50% 
correct  case  makers  as  presented  in  Table2. 


Table  2.  Performance  in  the  case  selection  task 


Correct 

Incorrect 

Total 

Responses 

Responses 

50% 

50% 

100% 

(20/40) 

(20/40) 

(40  cases) 

After  selecting  the  case  markers,  the  patient  was  asked  to  judge  whether  the  sentences  filled  by  wrong 
and  correct  case  markers  are  grammatical  or  ungrammatical.  Interestingly,  the  patient  judged  the 
sentences  as  the  grammatical  sentences,  even  though  those  sentences  are  ungrammatical.  According  to  the 
results  of  the  case  selection  task,  the  Broca  patient  made  many  errors  in  selecting  case  markers  and  filling 
in  the  empty  blanks(50%).  The  dissociation  in  comprehending  and  producing  the  subjective  and  objective 
case  markers  found  in  the  sentence-picture  matching  and  the  case  selection  task  implicates  that  the 
syntactic  processes  involved  in  comprehending  and  producing  the  case  postpositions  are  separated  and 
modular  in  the  respects  of  language  processing  or  structure. 

In  order  to  reevaluate  the  validity  of  the  case  selection  task  and  in  order  to  see  whether  producing  the 
subject  case  is  different  from  generating  the  object  case,  the  case  generation  task  was  employed.  One 
example  sentence  presented  to  the  patient  in  the  case  generation  task  is  shown  at  the  below.  The  patient 
task  was  to  generate  an  appropriate  case  marker  without  any  alternatives. 

°1  ^1  c~l  k  ^(in  English,  which  means  that  "the  president  received  a 

phone  call  yesterday".). 

The  correct  answers  at  the  above  blanks  are  the  subjective  case  marker(for  the  1st  blank),  the 
objective  case  marker(for  the  2nd  blank),  and  the  suffix  to  represent  the  past  tense(for  the  3rd  blank).  The 
1st  and  2nd  blanks  are  to  generate  the  case  markers  and  the  3rd  blank  is  to  produce  the  suffix( which  is 
called  “dd°1  s:0! D] ’’(which  suffixes  are  used  to  express  the  tense  and  to  enrich  the  meaning  of  the  word 
stem)  and  is  placed  before  the  terminal  suffix( which  is  called  “ojfrhojn]”  and  is  used  to  terminate  the 
sentence).  In  the  current  study,  two  kinds  of  “'id  s'0! u]  ”  were  evaluated.  One  is  to  evaluate  the  tense 
and  the  other  was  to  see  the  honorific  “dd'H  The  performance  of  the  Broca  patient  in  the  case 

generation  task  was  shown  in  the  table  3.  As  found  in  the  case  selection  task,  the  Broca  patient  generated 
the  wrong  case  markers  around  50%,  which  suggests  that  the  agrammatic  Broca  patient  has  difficulty  in 
generating  case  marker.  Also,  it  was  found  that  the  performance  in  producing  correctly  the  subject  case 
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markers  was  much  lower  than  that  in  the  objective  case.  And  the  patient  produced  the  wrong  case 
postpositions  about  50%.  He  made  errors  about  50%  for  the  tense  generation,  whereas  he  did  not  make 
many  errors  in  the  honorific  n]  ”  generation. 


Table  3.  Performance  in  the  case  generation  task 


Correct 

Response 

Incorrect 

Response 

Context- 

Inappropriate 

Response 

Total 

Subjective 

Case 

(6/23) 

(14/23) 

(3/23) 

23 

Objective 

Case 

(17/22) 

(4/22) 

(1/22) 

22 

Tense 

Suffix 

4 

4 

4 

12 

Aspect 

Suffix 

5 

1 

1 

7 

This  fact  provides  the  double  dissociation  in  considering  the  previous  studies(Kim,  1997;  Lee,  1998). 
That  is,  Kim(1997)  and  Lee(1998)  reported  that  two  Broca  patients  showed  superior  performance  in 
producing  the  objective  cases  to  the  performance  in  the  subjective  cases.  Thus,  the  current  and  the 
previous  studies  suggest  that  the  different  case  postpositions  are  processed  differently  and  producing  the 
different  case  markers  are  performed  in  the  modular  fashion.  One  hypothesis  to  explain  this  double 
dissociation,  which  is  named  VP-internal  subject  hypothesis,  is  to  assume  that  the  subject  case  is 
governed  and  role-assigned  by  the  tense  of  the  main  verb  and  the  object  case  is  determined  by  the  verb. 

Also,  the  single  dissociation  in  inflection  for  tense  and  honorific  "dd'H  U1 "  was  found.  Based  on 
this  single  dissociation,  it  is  possible  that  the  inflections  for  the  tense  and  aspect  might  be  modular  in  the 
respects  of  the  functional  localization  or  information  processing. 
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The  Neuroanatomy  of  Sentence  Comprehension 
Edgar  Zurif 

Volen  Center  for  Complex  System,  Brandeis  University 

My  research  partners  and  I  study  the  relation  between  brain  and  language.  We  focus  primarily  on  the 
neuroanatomical  organization  of  the  system  for  natural  language  comprehension.  To  establish  this 
perspective,  we  rely  on  data  from  aphasia  research  and,  more  recently,  on  functional  neuroimaging 
data. 

APHASIA  RESEARCH 

Our  aphasia  research  is  guided  by  a  functional  analysis  of  the  normalsystem  -  a  description  of  some  of 
its  components  (broadly,  those  having  to  do  with  syntactic  and  semantic  processing)  and  of  the  way  in 
which  these  components,  or  modules,  interact.  This  much  is  standard.  Also  standard:  We  take  the  specific 
deficits  following  focal  brain  damage  to  be  explicable  in  terms  of  disruptions  to  one  or  more  these 
components. 

What  sets  us  somewhat  apart  from  many  other  researchers,  however,  are  the  kinds  of  details  that  enter 
into  our  componential  analysis  and  that  support  our  inferences  concerning  "functional  lesions."  The 
difference  turns  on  the  way  in  which  we  isolate  modules.  Unlike  other  approaches,  we  do  not  license 
processing  modules  solely  on  the  basis  of  formal  linguistic  theory.  We  do  not  distinguish,  say,  between  a 
syntactic  processing  constituent  and  a  semantic  processing  constituent  simply  because  the 
representational  formats  they  operate  on  are  different.  Rather,  we  also  seek  evidence  for  the  uniqueness  of 
processing  constituents  in  terms  of  their  real-time  fixed  and  mandatory  operating  characteristics. 

The  point  here  is  that  modules  and  their  representations  constitute  evanescent,  intermediate  stages  in 
the  chain  of  comprehension;  they  are,  therefore,  most  directly  revealed  by  measurements  taken  during  the 
brief  course  of  their  operation.  And  in  line  with  this  notion,  we  chart  "functional  lesions"  in  terms  of 
alterations  to  their  operating  characteristics.  In  effect,  we  seek  a  functional  layout  of  the  comprehension 
system  that  is  elaborated  in  real-time  terms.  That's  one  feature  of  our  research  program. 

Another  feature  turns  on  neurological  matters.  In  the  first  instance,  we  try  to  provide  evidence  that  the 
effects  of  brain  damage  distinguish  between  modules,  sparing  one,  disrupting  another.  In  this  way  we 
check  whether  our  theory  is  neurologically  defensible.  But  we  seek  more:  We  also  seek  data  on  the 
neuroanatomical  layout  of  the  system  -  on  how  the  modules  are  geographically  distributed. 

Our  connection  to  neuroanatomy  is  based  on  the  fact  that  the  aphasic  syndromes  we  study  -  Broca’s 
and  Wernicke's  for  the  most  part  -  are  distinguishable  both  clinically  and  with  respect  to  lesion  site.  So, 
clinically,  we  can  contrast,  among  many  other  things,  the  nonfluent  telegraphic  speech  of  Broca's  aphasic 
patients  and  the  fluent,  rather  empty  speech  of  Wernicke's  aphasic  patients  (Goodglass  &  Kaplan,  1972). 
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And  neuroanatomically,  we  can  contrast  the  two  syndromes  along  an  anterior  posterior  axis.  Although 
variable,  the  generally  large  anterior  lesions  associated  with  Broca's  aphasia  cluster  about  a  modal  site 
that  is  quite  different  from  that  for  Wernicke's  aphasia.  For  the  latter,  the  greatest  involvement  appears  to 
be  confined  to  more  posterior  regions,  implicating  especially  the  posterior  superior  portion  of  the  first 
temporal  gyrus.  (For  details  see,  e.g.,  Alexander  et  al.,  1990;  Benson,  1985;  Naeser  et  al.,  1989;  Vignolo, 
1988) 

We  capitalize  on  this  difference.  Our  work  shows  that  the  brain  area  implicated  in  Broca's  aphasia,  but 
not  Wernicke’s,  is  necessary  for  the  real-time  formation  of  syntactically-licensed  dependency  relations. 
These  are  linking  operations  that  allow  the  listener  to  identify  the  referents  of  pronouns  and  reflexives, 
and  more  abstractly,  to  interpret  constituents  in  non-canonical  positions  -  to  determine  for  example,  that 
"the  boy"  in  "It  was  the  boy  that  the  girl  chased"  is  to  receive  its  argument  as  the  direct  object  of  "chased", 
even  though  it  is  near  the  beginning  of  the  sentence  instead  of  after  the  verb.  Further,  our  work  connects 
such  operations  to  timing  parameters,  thereby  suggesting  that  the  brain  region  marked  by  Broca's  aphasia 
is  involved  in  syntactic  processing,  not  because  it  is  the  locus  of  syntactic  representations  per  se,  but 
because  it  provides  necessary  elementary  processing  resources  -  resources  that  intuitively  appear  to  be 
biologically  fixed  or  "wired  in.” 

That  is  the  first  half  of  a  double  dissociation.  The  second  half  is  based  on  our  demonstration  that  the 
region  associated  with  Wernicke's  aphasia,  but  not  Broca's,  is  crucially  involved  in  semantic 
combinatorial  operations.  Such  operations  are  required  when  semantic  composition  is  not  syntactically 
transparent,  as  in  the  sentence,  "The  girl  jumped  until  dawn"  where  somehow  (either  by  type-shifting  or 
interpolation)  the  sense  of  repetition  -  of  repeated  jumping  -  must  be  generated  of  pasted  into  the 
interpolation  as  comprehension  unfolds  in  real-time. 

FUNCTIONAL  NEUROIMAGING 

Imaging  studies  of  comprehension  in  neurologically  intact  subjects  both  support  and  refine  the  cortical 
localization  patterns  observed  in  aphasia  research.  So  in  a  recent  study  carried  out  with  colleagues  at  The 
University  of  Pennsylvania,  we  observed  recruitment  of  left  inferior  frontal  cortex  (Broca's  area) 
specifically  for  the  comprehension  of  non-canonical  sentences  in  which  the  syntactically-licensed 
dependency  relations  had  to  be  established  over  relatively  long  "distances."  By  contrast,  no  such  syntactic 
specialization  was  shown  by  a  core  region  of  left  posterior  superior  temporal  cortex  (Wernicke's  area). 
What  we  are  now  trying  to  determine  is  whether  Wernicke's  are  is  recruited  for  compositional  processing 
that  is  entirely  semantic.  Evidence  -  emerging  from  a  study  that  we  are  carrying  out  in  collaboration  with 
researchers  at  McLean  Hospital  -  is  beginning  to  suggest  this  possibility.  But  this  work  is  just  starting  and 
the  evidence  is  preliminary. 
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Linking  visual  attention  and  awareness  with  functional  MRI 

Geraint  Rees 

Division  of  Biology  139-74,  California  Institute  of  Technology 

What  we  see  is  not  dictated  solely  by  where  we  point  our  eyes,  but  also  by  how  we  direct  our  visual 
attention.  The  effects  of  directed  attention  on  visual  perception  can  be  profound.  While  attending  to  a 
stimulus  can  substantially  enhance  a  subject’s  capacity  to  perform  fine  discriminations,  an  ignored 
stimulus  sometimes  appears  not  even  to  reach  awareness.  However,  the  nature  and  extent  of  processing  for 
attended  versus  ignored  stimuli  has  been  a  subject  of  controversy  for  many  decades,  and  the  brain 
mechanisms  leading  to  such  effects  have  until  recently  been  unclear.  This  presentation  will  consider  our 
recent  investigations  of  the  nature  and  extent  of  processing  for  unattended  stimuli  using  functional 
magnetic  resonance  imaging  (fMRI)  in  humans. 

Cortical  processing  of  ignored  stimuli  depends  on  perceptual  load 

Observations  in  monkeys  suggest  that  attention  can  modulate  neural  responses  to  stimuli  in 
functionally  specialised  areas  of  the  visual  system.  To  investigate  this  in  humans,  subjects  were  scanned 
with  fMRI  during  repeated  presentations  of  identical  visual  motion  stimuli  while  only  the  attentional 
component  of  the  task  was  varied  (1).  Enhanced  haemodynamic  responses  during  attentive  conditions 
defined  an  occipito-parieto-frontal  network,  including  sensory  and  association  areas.  Attentional 
modulation  was  not  restricted  to  extrastriate  areas  (including  V3a  and  the  V5  complex)  but  was  also 
evident,  to  a  lesser  degree,  in  early  visual  areas  close  to  the  calcarine  fissure. 

These  findings  show  that  attending  to  a  stimulus  enhances  processing  of  relevant  stimuli  in  extrastriate 
cortex  compared  to  a  passive  viewing  baseline.  However,  as  the  unattended  stimulus  is  not  manipulated 
independently  of  attention,  it  is  not  possible  to  conclude  whether  the  effects  observed  on  cortical  activity 
represent  the  effects  of  attention,  those  of  unattended  stimulus  processing,  or  an  interaction  of  the  two. 
This  issue,  the  extent  to  which  unattended  stimuli  are  processed,  is  one  of  the  longest  standing 
controversies  in  the  psychology  of  attention.  While  some  have  suggested  that  unattended  stimuli  receive 
only  a  cursory  processing  of  elementary  physical  attributes,  others  have  concluded  that  unattended  stimuli 
are  fully  processed. 

Lavie's  theory  of  attention  proposes  a  resolution  to  this  longstanding  controversy  by  suggesting  that  the 
processing  load  in  a  relevant  task  determines  the  extent  to  which  irrelevant  distractors  are  processed.  This 
theory  was  tested  by  asking  participants  in  a  study  to  perform  linguistic  tasks  of  low  or  high  load  while 
ignoring  irrelevant  visual  motion  in  the  periphery  of  the  display  (2).  Although  task  and  distractor  were 
unrelated,  both  functional  imaging  of  motion-related  activity  in  cortical  area  V5  and  psychophysical 
measures  of  the  motion  after-effect  showed  reduced  motion  processing  during  high  load  in  the  linguistic 
task.  These  findings  fulfil  the  prediction  that  perception  of  irrelevant  distractors  depends  on  the  relevant 
processing  load.  Moreover,  an  interesting  implication  of  these  findings  is  that  ignoring  an  irrelevant 
stimulus  is  not  under  voluntary  control,  but  instead  depends  critically  on  perceptual  load  in  the  relevant 
task. 


Neural  systems  for  attention  and  visual  awareness  are  linked 
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The  effects  of  attention  on  brain  responses  described  above  included  not  just  modulation  of  sensory 
processing,  but  also  delineated  a  parietofrontal  network  that  has  been  repeatedly  implicated  in  both  spatial 
and  non-spatial  attention.  Is  this  network  specific  to  the  control  of  attention,  or  does  it  subserve  some 
more  general  function? 

We  investigated  this  by  studying  the  neural  basis  of  perceptual  transitions  during  binocular  rivalry  (3). 
Rivalry  occurs  when  dissimilar  images  are  presented  to  the  two  eyes.  Perception  alternates 
spontaneously  between  each  monocular  view  and  is  not  under  voluntary  control.  Functional  brain  imaging 
in  humans  was  used  to  study  the  neural  basis  of  these  subjective  perceptual  changes.  Cortical  regions 
whose  activity  reflected  perceptual  transitions  included  extrastriate  areas  of  the  ventral  visual  pathway, 
and  parietal  and  frontal  regions,  all  of  which  have  been  implicated  in  spatial  attention.  However,  whereas 
the  extrastriate  areas  were  also  engaged  by  nonrivalrous  perceptual  changes,  activity  in  the  frontoparietal 
cortex  was  specifically  associated  with  perceptual  alternation  only  during  rivalry.  The  location  of  the 
frontoparietal  areas  was  strongly  overlapping  with  areas  previously  implicated  in  the  control  of  visual 
attention.  This  suggests  that  frontoparietal  areas  play  a  central  role  in  conscious  perception,  biasing  the 
content  of  visual  awareness  toward  abstract  internal  representations  of  visual  scenes,  rather  than  simply 
toward  space.  One  possibility  is  that  right  frontoparietal  structures  perform  a  general  selective  function, 
choosing  between  alternative  interpretations  of  the  visual  scene  competing  for  awareness  in  the  ventral 
stream. 

Inattentional  blindness  occurs  under  conditions  of  high  perceptual  load 

As  has  already  been  alluded  to,  whether  the  identity  of  meaningful  stimuli  is  still  extracted  when 
ignored  is  a  classic  issue  in  attention  research.  Recent  studies,  including  the  data  presented  here,  show  that 
visual  attention  can  modulate  stimulus-evoked  activity  in  early  visual  brain  areas.  However,  the  stimuli 
used  were  typically  meaningless  (patches  or  moving  dots),  so  cannot  resolve  the  issue  for  meaningful 
stimuli  such  as  ignored  words.  Furthermore,  if  the  response  to  an  unattended  stimulus  is  merely  attenuated 
rather  than  eliminated  at  early  levels  of  processing,  it  might  continue  through  to  the  level  of  meaning.  A 
hybrid  theory  of  attention  such  as  Lavie’s  predicts  that  processing  of  higher  order  cognitive  properties  will 
be  suppressed  under  conditions  of  high  perceptual  load.  We  therefore  constructed  a  situation  where 
subjects  could  look  directly  at  a  word  without  attending  to  it,  under  conditions  of  high  perceptual  load  (4). 
We  measured  brain  activity  with  fMRI,  in  humans  viewing  rapid  successive  displays  of  overlapping 
pictures  plus  letter-strings,  which  could  be  meaningful  words  or  non-words.  Directing  attention  to  letter- 
strings  versus  pictures  strongly  modulated  activity  in  extrastriate  cortex,  despite  the  spatial  overlap  of  the 
stimuli.  Cortical  regions  whose  activity  reflected  word  identity  of  letter-strings  when  attended  included 
areas  of  left  temporal,  parietal  and  frontal  cortex  classically  associated  with  language.  Critically,  when  the 
letter-strings  were  ignored,  the  activity  related  to  word  identity  was  abolished.  These  findings  demonstrate 
'inattentional  blindness'  even  for  highly  familiar  and  foveated  stimuli.  We  suggest  that  subjects  fail  to 
remember  these  unattended  stimuli  due  to  the  absence  of  brain  activity  reflecting  their  identity  at  the  time 
of  presentation. 

Unconscious  activation  of  VI  in  parietal  extinction  by  an  unseen  stimulus. 

The  findings  presented  above  imply  a  close  link  between  the  availability  of  visual  attention,  cortical 
processing  of  unattended  stimuli  and  visual  awareness.  Furthermore,  damage  to  frontoparietal  structures 
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results  in  disorders  of  attention  and  awareness.  Visual  extinction  is  a  sign  classically  associated  with  right 
parietal  damage.  The  patient  can  see  a  single  stimulus  presented  in  the  ipsilesional  or  contralesional  visual 
field,  but  is  characteristically  unaware  of  the  same  contralesional  stimulus  during  simultaneous  stimulation 
of  both  fields.  The  ipsilesional  stimulus  is  said  to  “extinguish”  the  contralesional  stimulus  from  awareness 
during  bilateral  stimulation,  perhaps  due  to  a  pathological  bias  in  attention  towards  the  ipsilesional  side. 
Recent  psychophysical  evidence  suggests  that,  although  extinguished  stimuli  are  not  consciously  seen, 
they  may  undergo  residual  processing  and  exert  implicit  effects  on  performance.  However,  the  neural 
structures  mediating  such  residual  processing  for  extinguished  stimuli  remain  unknown.  We  therefore 
studied  the  neural  activity  evoked  by  an  extinguished  visual  stimulus  using  event-related  fMRI,  in  a 
patient  with  circumscribed  right  inferior  parietal  damage  and  profound  left-sided  extinction  (5). 
Monochrome  objects  (faces  or  houses)  were  presented  in  the  left  or  right  field,  either  unilaterally  or 
bilaterally  on  each  trial,  with  the  patient  indicating  by  button  press  whether  he  saw  an  object  on  the  left, 
the  right,  or  on  both  sides.  He  usually  saw  only  the  right  object  on  bilateral  trials,  yet  the  fMRI  data 
showed  activation  of  visual  cortex  contralateral  to  the  extinguished  left  stimulus  on  these  trials  (as 
compared  with  right-only  stimulation),  in  both  striate  and  early  extrastriate  areas  of  the  right  hemisphere. 
This  activity  had  a  similar  location  and  timecourse  to  that  resulting  from  a  single  stimulus  in  the  left 
versus  right  visual  field.  Cortical  pathways  involved  in  the  normal  processing  of  a  single  seen  stimulus  can 
thus  still  be  activated  by  an  unseen,  extinguished  stimulus  after  right  parietal  damage.  This  in  turn 
suggests  that  activation  of  primary  visual  cortex  may  not  be  sufficient  to  evoke  awareness. 
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There  is  substantial  evidence  that  the  parietal  lobes  are  involved  in  locating  items  in  space.  In 
humans  bilateral  parietal  lobe  damage  can  result  in  a  neuropsychological  disorder  known  as  Balint’s 
syndrome  in  which  single  shapes  that  define  objects  are  perceived  but  their  location  is  not  explicitly 
known.  Recent  evidence  also  demonstrates  that  surface  features  such  as  color  and  size  are 
abnormally  bound  with  shape  after  parietal  damage.  This  finding  supports  the  role  of  parietal 
function  in  co-locating  two  features  for  the  purpose  of  feature  integration.  Features  such  as  color, 
size  and  shape  are  represented  in  ventral  areas  of  the  temporal  lobe  (the  ventral  what  pathway), 
while  location  is  associated  with  the  parietal  and  frontal  lobes  (the  dorsal  where  pathway ).  The 
data  reported  in  the  present  paper  demonstrate  that  although  distinctive  features  may  not  be  located 
accurately  when  bilateral  parietal  lobe  damage  occurs,  their  spatial  location  is  represented,  albeit 
below  the  level  of  awareness.  The  results  indicate  that  spatial  location  of  features  is  coded  even 
when  both  parietal  lobes  are  damaged.  They  also  support  recent  evidence  for  multiple  spatial  maps 
in  other  primates. 


It  has  long  been  known  that  parietal  lobe  damage  in  humans  can  result  in  severe  spatial  disorientation 
(De  Renzi,  1986),  with  more  recent  accounts  suggesting  a  deficit  in  spatial  attention  (Posner,  Walker, 
Friedrich,  &  Rafal,  1984)  or  spatial  representation  (Bisiach,  Capitani,  Luzzatti,  &  Perani,  1981).  However, 
there  appear  to  be  multiple  spatial  maps  in  the  primate  brain  that  are  widely  distributed  across  cortical  and 
subcortical  areas  (Graziano  &  Gross,  1996;  Stein,  1992).  What  spatial  information  is  necessary  for 
attentional  control  and  spatial  awareness  associated  with  parietal  function  and  what  spatial  information 
can  be  encoded  in  spatial  maps  when  conscious  access  is  denied,  as  in  the  case  of  parietal  damage 
(Robertson,  Treisman,  Friedman-Hill,  &  Grabowecky,  1997). 

Evidence  for  multiple  spatial  maps  has  taken  on  added  importance  in  light  of  recent  findings 
demonstrating  that  the  disruption  of  spatial  abilities  due  to  damage  to  both  parietal  lobes  also  disrupts  the 
ability  to  correctly  bind  features  such  as  color  and  shape  and  size  and  shape  (Friedman-Hill,  Robertson,  & 
Treisman,  1995).  This  binding  problem  in  turn  makes  it  very  difficult  to  accurately  search  a  cluttered 
array  for  the  conjunction  of  two  features.  For  instance,  searching  for  a  red  X  among  red  Os  and  green 
Xs  may  be  impossible  even  in  the  presence  of  only  a  few  distractors.  Without  conscious  access  to  an 
adequate  spatial  representation  of  the  external  world  to  guide  spatial  attention,  features  of  objects  such  as 
shape  and  color  seem  to  migrate  to  produce  illusory  conjunctions  in  perception. 

Other  spatial  maps  associated  with  the  temporal  lobes  has  been  suggested  on  the  basis  of  animal 
research,  but  if  they  exist  in  humans,  neuropsychological  observation  suggests  that  they  are  not  sufficient 
to  support  the  normal  perception  of  correctly  bound  features  or  their  location..  The  present  paper  reports 
data  collected  with  a  patient  with  bilateral  parietal  lobe  lesions  and  Balint’s  syndrome  (R.M.)  in  whom 
feature  binding  and  location  deficits  were  reported  previously  (Friedman-Hill,  et.  al.,  1995).  The  present 
study  first  shows  that  feature  binding  deficits  were  still  present  in  this  patient  and  could  be  observed 
under  the  conditions  of  this  experiment.  This  study  then  demonstrates  that  location  information  exists 
in  R.M.’s  visual  system  below  the  level  of  awareness  and  verifies  that  a  feature  can  be  perceived  without 
explicit  awareness  of  its  location. 

Balint's  syndrome  is  fortunately  quite  rare,  as  it  renders  a  person  functionally  blind  in  its  classical 
form.  The  patients  typically  see  only  one  shape  and  can  report  what  it  is,  but  they  do  not  know  where  it 
is.  They  cannot  accurately  reach  for  it  or  track  it  despite  intact  primary  motor  abilities.  This  problem 
disrupts  activities  of  every  day  living  such  as  navigation,  dressing,  watching  TV,  reading,  eating,  etc.,  and 
leaves  the  patient  with  very  few  options.  Language  and  other  cognitive  abilities  remain  intact.  R.M. 
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presented  with  the  classic  Balint's  syndrome  just  described  in  1992  but  has  improved  somewhat  over 
time. 


Figure  1.  The  sequence  of  displays  in  feature  search  with  a  central  arrow  probe. 

An  indirect  measure  was  used  to  test  whether  or  not  spatial  information  was  coded  implicitly.  Using 
a  probe  technique  and  a  dual  task,  we  presented  a  4  item  feature  display  containing  1  green  circle  target 
and  3  red  circle  distractors  along  with  a  central  fixation  pattern  that  changed  into  an  arrow  at  variable 
times  after  the  feature  display  appeared  (see  Figure  1).  Instructions  were  to  press  a  response  key  as 
rapidly  as  possible  when  an  arrow  and  to  withhold  a  response  if  the  fixation  pattern  changed  to  anything 
other  than  an  arrow.  After  their  response  participants  were  to  report  whether  the  target  had  been  present 
or  not  in  the  feature  display  by  saying  yes  or  no. 

Experiments  with  young  normal  subjects  showed  that  response  times  to  the  arrow  were  faster  when 
the  arrow  pointed  to  the  target  location  than  when  it  pointed  to  one  of  the  distractor  locations  even  though 
neither  the  direction  of  the  arrow  nor  the  location  of  the  search  item  was  necessary  to  perform  either  task 
(Fig.  2a).  These  results  demonstrate  that  the  direction  of  the  arrow  probe  and  its  spatial  relationship  to 
the  location  of  the  target  is  encoded  (probably  automatically). 

Despite  his  spatial  deficits,  R.M.  showed  a  similar  pattern  of  results  as  normal  subjects:  when  the 
arrow  pointed  to  the  target,  he  detected  the  arrow  faster  than  when  it  pointed  to  one  of  the  distractors. 
Surprisingly,  RM.  showed  this  probe  RT  pattern  even  when  his  ability  to  detect  the  feature  target  was 
very  poor  (R.M.  produced  28%  hits  and  15%  false  positives,  while  normal  subjects  produced  98%  hits 
and  less  than  3%  false  positives)  (Fig.  2b).  Although  R.M.  showed  great  difficulty  in  correctly  reporting 
the  presence  of  the  feature  target  in  this  experiment,  it  may  have  been  because  the  target  faded  rapidly 
from  memory  or  that  the  dual  task  directly  interfered  with  the  perceptual  registration  of  the  target. 
Whatever  the  reason  for  the  difficulty  in  feature  detection  accuracy,  it  is  clear  that  the  target’s  location 
and  its  spatial  relationship  to  the  orientation  of  the  arrow  continued  to  exert  control. 
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a 

Young  Normal  (N=13) 


Location  to  which  Arrow  Probe  is  pointing 


b 

R.  M.  (8  Sessions) 


Location  to  which  Arrow  Probe  is  pointing 


Figure  2.  RT  results  with  young  normal  subjects  (a)  and  R.M.  (b).  The  probe  RTs  were  organized  by  the 
direction  of  the  arrow,  relative  to  the  target  location. 

To  address  the  question  of  whether  the  spatial  relationship  between  the  arrow  and  the  feature  was 
explicitly  known,  we  presented  the  arrow  and  the  target  display  simultaneously  and  they  remained  on  the 
screen  until  R.M.  responded.  In  this  study,  he  was  simply  asked  to  report  whether  the  arrow  pointed  at 
a  target  or  one  of  the  circle  distractors.  Only  target  present  displays  were  used.  He  was  unable  to  do 
this  task  above  chance  (51%  accurate).  He  also  reported  that  this  was  a  difficult  task  for  him  and 
shook  his  head  side  to  side  while  performing  the  task.  It  should  be  noted  that  this  was  not  due  to  spatial 
confusions  per  se.  R.M.  had  full  spatial  awareness  of  the  location  of  his  own  body  parts  and  the  spatial 
relationship  between  them  even  when  his  Balint’s  symptoms  were  more  severe  than  at  the  time  of  this 
study. 

These  results  support  the  existence  of  separate  spatial  maps  within  the  human  brain  that  remain  intact 
even  when  both  parietal  lobes  are  damaged.  Furthermore,  they  demonstrate  that  intact  parietal  lobes  are 
necessary  to  explicitly  access  and  utilize  this  information  for  the  purpose  of  guiding  attention  or  action  to 
a  location.  In  turn,  attention  to  a  location  is  necessary  to  properly  bind  features  such  as  shape  and  color 
in  cluttered  arrays.  The  present  findings  are  also  consistent  with  electrophysiological  data  in  monkeys 
for  the  existence  of  multiple  spatial  representations.  The  evidence  presented  here  supports  these 
observations  but  also  demonstrates  that  some  spatial  information  can  be  coded  without  awareness  and  is 
not  accessible  without  intact  parietal  lobes. 
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